SUPPORT / SAMPLES & SAS NOTES
Problem Note 63226: The SAS® importpackage causes the SAS® Metadata Server cluster node to fail with "AccessViolation"
 |  |  |  |
When you import a complex package, the master metadata server node might fail with an AccessViolation in the metadata server log, as shown below:
2018-09-06T14:15:47,890 ERROR [00000041] :sas - AccessViolation (6) encountered in CONTEXT 00000028 using cookie 5.
In the tk.report.##### log, the following might appear:
ERROR: The system has encountered an unhandled Exception.
: Please contact technical support and provide them
: with the following traceback and system dump information:
ABORT: SAS/TK is aborting
: Abort Log file [tk.reportlog.12126]
: ---------------------------------------
: Time [9/06/2018 2:15:48 PM]
: ---------------------------------------
: Thread Name [tkoms cluster manager thread]
: Please contact technical support and provide them
: with the following traceback and system dump information:
ABORT: SAS/TK is aborting
: Abort Log file [tk.reportlog.12126]
: ---------------------------------------
: Time [9/06/2018 2:15:48 PM]
: ---------------------------------------
: Thread Name [tkoms cluster manager thread]
Here is the traceback:
/94m5/install/SASFoundation/9.4/sasexe/tkmk.so(bkt_abort_tkt_cb+0xb6)[0x7ff4baf8b306]
/94m5/install/SASFoundation/9.4/sasexe/tkmk.so(skaCallAbortRoutines+0xf0)[0x7ff4baf8a2a0]
/94m5/install/SASFoundation/9.4/sasexe/tkmk.so(bkabort+0x99) [0x7ff4baf95679]
/94m5/install/SASFoundation/9.4/sasexe/tkmk.so(bkt_signal_handler+0x144)[0x7ff4baf87a44]
/lib64/libpthread.so.0(+0xf5e0) [0x7ff4bc5a55e0]
/94m5/install/SASFoundation/9.4/sasexe/tkoms.so(__intel_ssse3_rep_memcpy+0x1ac7)[0x7ff48e237467]
/94m5/install/SASFoundation/9.4/sasexe/tkoms.so(FOMSclustmgrTransferData+0x32ef)[0x7ff48e1c0d2f]
/94m5/install/SASFoundation/9.4/sasexe/tkoms.so(+0x3bbccc) [0x7ff48e1acccc]
/94m5/install/SASFoundation/9.4/sasexe/tkmk.so(sktMain+0x94) [0x7ff4baf85494]
/94m5/install/SASFoundation/9.4/sasexe/tkmk.so(bktMain+0x71) [0x7ff4baf87231]
/lib64/libpthread.so.0(+0x7e25) [0x7ff4bc59de25]
/lib64/libc.so.6(clone+0x6d) [0x7ff4bbb8e34d]
/94m5/install/SASFoundation/9.4/sasexe/tkmk.so(skaCallAbortRoutines+0xf0)[0x7ff4baf8a2a0]
/94m5/install/SASFoundation/9.4/sasexe/tkmk.so(bkabort+0x99) [0x7ff4baf95679]
/94m5/install/SASFoundation/9.4/sasexe/tkmk.so(bkt_signal_handler+0x144)[0x7ff4baf87a44]
/lib64/libpthread.so.0(+0xf5e0) [0x7ff4bc5a55e0]
/94m5/install/SASFoundation/9.4/sasexe/tkoms.so(__intel_ssse3_rep_memcpy+0x1ac7)[0x7ff48e237467]
/94m5/install/SASFoundation/9.4/sasexe/tkoms.so(FOMSclustmgrTransferData+0x32ef)[0x7ff48e1c0d2f]
/94m5/install/SASFoundation/9.4/sasexe/tkoms.so(+0x3bbccc) [0x7ff48e1acccc]
/94m5/install/SASFoundation/9.4/sasexe/tkmk.so(sktMain+0x94) [0x7ff4baf85494]
/94m5/install/SASFoundation/9.4/sasexe/tkmk.so(bktMain+0x71) [0x7ff4baf87231]
/lib64/libpthread.so.0(+0x7e25) [0x7ff4bc59de25]
/lib64/libc.so.6(clone+0x6d) [0x7ff4bbb8e34d]
If you start the old master node, it might fail and stop responding. If so, kill the metadata server with signal 9:
In the new SASMeta_MetadataServer_2018-09-06_trcv198_13377:
2018-09-06T14:53:34,943 INFO [00000013] :sas - NOTE: File I0000002.PROP.DATA is damaged.
2018-09-06T14:53:34,958 INFO [00000013] :sas - NOTE: Data set I0000002.PROP contained structural errors which were fixed; however some changes during last
update may not have been written to disk. The following messages indicate type of structural error repaired.
and
2018-09-06T14:53:39,610 ERROR [00000008] :sas - Expansion creates longer string than output area given.
2018-09-06T14:53:39,610 ERROR [00000008] :sas - CRITICAL ERROR encountered reading journal entry at file position 1024.
2018-09-06T14:53:39,610 ERROR [00000008] :sas - Crash recovery abandoned.
2018-09-06T14:53:39,611 ERROR [00000008] :sas - IOM call failed. Internal server exception: access violation.
2018-09-06T14:53:34,943 INFO [00000013] :sas - NOTE: File I0000002.PROP.DATA is damaged.
2018-09-06T14:53:34,958 INFO [00000013] :sas - NOTE: Data set I0000002.PROP contained structural errors which were fixed; however some changes during last
update may not have been written to disk. The following messages indicate type of structural error repaired.
and
2018-09-06T14:53:39,610 ERROR [00000008] :sas - Expansion creates longer string than output area given.
2018-09-06T14:53:39,610 ERROR [00000008] :sas - CRITICAL ERROR encountered reading journal entry at file position 1024.
2018-09-06T14:53:39,610 ERROR [00000008] :sas - Crash recovery abandoned.
2018-09-06T14:53:39,611 ERROR [00000008] :sas - IOM call failed. Internal server exception: access violation.
Workaround:
Import simple and quick metadata rather than complex metadata. The recommendation is to import less than 45 seconds worth of metadata.
Do not import complex packages into a cluster, but Import only in single metadata server mode ( -startNoCluster ).
If this problem occurs, complete these steps:
Note: The two remaining metadata server slaves should reconstitute a quorum.
- Click Run Backup Now from SAS® Management Console.
- On the failed node, use "./MetadataServer.sh -recover" to restore from the latest backup.
This issue is fixed in SAS® 9.4M6 (TS1M6). The fix stops the import package if it is too complex for the metadata server. In this scenario, instead of an AccessViolation, the following message is reported:
INFO - ******** Importing Metadata ********
ERROR - Error importing metadata. Reason: This request requires changing too
much metadata to be safely committed. Please try again with a smaller request.
INFO - The import process has finished with errors.
ERROR - Error importing metadata. Reason: This request requires changing too
much metadata to be safely committed. Please try again with a smaller request.
INFO - The import process has finished with errors.
Operating System and Release Information
| Product Family | Product | System | Product Release | SAS Release | ||
| Reported | Fixed* | Reported | Fixed* | |||
| SAS System | SAS Metadata Server | Microsoft Windows 8 Pro x64 | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 |
| Microsoft Windows 8 Pro 32-bit | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Microsoft Windows 8 Enterprise x64 | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Microsoft Windows 8 Enterprise 32-bit | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Microsoft® Windows® for x64 | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| z/OS | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Microsoft Windows 8.1 Enterprise 32-bit | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Microsoft Windows 8.1 Enterprise x64 | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Microsoft Windows 8.1 Pro 32-bit | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Microsoft Windows 8.1 Pro x64 | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Microsoft Windows 10 | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Microsoft Windows Server 2008 | 9.4_M3 | 9.4 TS1M3 | ||||
| Microsoft Windows Server 2008 R2 | 9.4_M3 | 9.4 TS1M3 | ||||
| Microsoft Windows Server 2008 for x64 | 9.4_M3 | 9.4 TS1M3 | ||||
| Microsoft Windows Server 2012 Datacenter | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Microsoft Windows Server 2012 R2 Datacenter | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Microsoft Windows Server 2012 R2 Std | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Microsoft Windows Server 2012 Std | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Windows 7 Enterprise 32 bit | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Windows 7 Enterprise x64 | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Windows 7 Home Premium 32 bit | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Windows 7 Home Premium x64 | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Windows 7 Professional 32 bit | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Windows 7 Professional x64 | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Windows 7 Ultimate 32 bit | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Windows 7 Ultimate x64 | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| 64-bit Enabled AIX | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| 64-bit Enabled Solaris | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| HP-UX IPF | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Linux for x64 | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||
| Solaris for x64 | 9.4_M3 | 9.4_M6 | 9.4 TS1M3 | 9.4 TS1M6 | ||